This paper presents media processor architectures for automotive applications. Media processing applications with their requirements for LSI implementations are first described for vision based driver assistance as well as graphical user interface for car navigation using 3D graphics. Then, parallel processing architectures for vision and graphics in these applications are reviewed with their performance and cost. After that, future trends of automotive media processing such as integration of vision and 3D graphics functions are shown with their applications and the required performance. Moreover, parallel processing architectures are discussed for the integration of vision and graphics. Finally, an prospect of a next-generation media processing LSI for automotives is provided.

Multi-projector technology has been under consideration in recent years. This technology allows the generation of wide field of view and high-resolution images in a cost-effective manner. It is expected to be applied extensively to training simulators where vivid immersive sensations and precision are required. However, in many systems the viewing frustums cannot be automatically assigned for distributed rendering, and the required manual setup is complicated and difficult. This is because the camera should be coincide exactly with a desired eye point to avoid perspective distortions. For the actual applications, the camera is seldom able to be set up at the desired eye point because of physical constraints, e.g., a narrow cockpit with many instruments. To resolve this issue, we have developed a "virtual camera method" that yields high-precision calibration regardless of the camera position. This method takes advantage of the quadratic nature of the display surface. We developed a practical real-time multi-projector display system for applications such as training simulators, that require high-accuracy in geometry and rapid response time.

A new and simple image processing approach for the measurement of the length of pedestrian crossings with a view to develop a travel aid for the blind people is described. In a crossing, the usual black road surface is painted with constant width periodic white bands. The crossing length is estimated using vector geometry from the left- and the right-border lines, the first-, the second- and the end-edge lines of the crossing region. Image processing techniques are applied on the crossing image to find these lines. Experimental results using real road scenes with pedestrian crossing confirm the effectiveness of the proposed method.

This paper newly proposes a method to automatically decompose real scene images into multiple object-oriented component regions. First, histogram patterns of a specific image feature, such as intensity or hue value, are estimated from image sequence and stored up. Next, Gaussian distribution parameters which correspond to object components involved in the scene are estimated by applying the EM algorithm to the accumulated histogram. The number of the components is simultaneously estimated by evaluating the minimum value of Bayesian Information Criterion (BIC). This method can be applied to a variety of computer vision issues, for example, the color image segmentation and the recognition of scene situation transition. Experimental results applied for indoor and outdoor scenes showed the effectiveness of the proposed method.

In the field of computer vision and computer graphics, Image-Based-Rendering (IBR) methods are often used to synthesize images from real scene. The image synthesis by IBR requires dense correct matching points in the images. However, IBR does not require 3D geometry reconstruction or camera calibration in Euclidean geometry. On the other hand, 3D reconstructed model can easily point out the occlusion in images. In this paper, we propose an approach to reconstruct 3D shape in a voxel space, which is named Projective Voxel Space (PVS). Since PVS is defined by projective geometry, it requires only weak calibration. PVS is determined by rectifications of the epipolar lines in three images. Three rectified images are orthogonal projected images of a scene in PVS, so processing about image projection is easy in PVS. In both PVS and Euclidean geometry, a point in an image is on a projection from a point on a surface of the object in the scene. Then the other image might have a correct matching point without occlusion, or no matching point because of occlusion. This is a kind of restriction about searching matching points or surface of the object. Taking advantage of simplicity of projection in PVS, the correlation values of points in images are computed, and the values are iteratively refined using the restriction described above. Finally, the shapes of the objects in the scene are acquired in PVS. The reconstructed shape in PVS does not have similarity to 3D shape in Euclidean geometry. However, it denotes consistent matching points in three images, and also indicates the existence of occluded points. Therefore, the reconstructed shape in PVS is sufficient for image synthesis by IBR.

Techniques for human-motion recovery are applicable to a variety of areas, such as sports, dancing, virtual reality, and video-game production. The people who work in this area focus their attention on recovering information on the motion of individuals rather than groups of people. It is important to demonstrate the possibility of recovering descriptions of the 3-D motion in team sports, since such information is able to provide us with a variety of information on the relations among players. This paper presents a new experimental result on 3-D motion recovery from a team sport. The result was obtained by a non-rigid shape recovery technique based on images from uncalibrated cameras. The technique was applied to recovering the 3-D motion of the players in a mini-basketball game which was played in a gymnasium. Some attention is focused on the analysis of the players' motion. Satisfactory results were obtained.

It has been over a decade since the "Eigenfaces" approach to automatic face recognition, and other appearance-based methods, made an impression on the computer vision research community and helped spur interest in vision systems being used to support biometrics and human-computer interface. In this paper I give a personal view of the original motivation for the work, some of the strengths and limitation of the approach, and progress in the years since. Appearance-based approaches to recognition complement feature- or shape-based approaches, and a practical face recognition system should have elements of both. Eigenfaces is not a general approach to recognition, but rather one tool out of many to be applied and evaluated in the appropriate context.

Recently, it becomes popular to synthesize new viewpoint images based on some sampled viewpoint images of real scene using technique of computer vision. 3D shape reconstruction in Euclidean space is not necessarily required, but information of dense matching points is basically enough to synthesize new viewpoint images. In this paper, we propose a new method for 3D reconstruction from three cameras based on projective geometry. In the proposed method, three input camera images are rectified based on projective geometry, so that the vertical and horizontal directions can be completely aligned with the epipolar planes between the cameras. This rectification provides Projective Voxel Space (PVS), in which the three axes are aligned with the directions of camera projection. Such alignment simplifies the procedure for projection between the 3D space and the image planes in PVS. Taking this advantage of PVS, silhouettes of the objects are projected into PVS, so that the searching area of matching points can be reduced. The consistency of color value between the images is also evaluated for final determination of the matching point. The finally acquired matching points in the proposed method are described as the surface of the objects in PVS. The acquired surface of the objects in PVS also includes knowledge about occlusion. Finally, images from new viewpoints can be synthesized from the matching points and occlusions. Although the proposed method requires only weak calibration, plausible occlusions are also synthesized in the images. In the experiments, images of virtual viewpoints, which were set among three cameras, are synthesized from three real images.

We describe a method for object recognition with 2D image queries to be identified from among a set of 3D models. The pose is known from a previous step. The main target application is face recognition. The 3D models consist of both shape and color texture information, and the 2D queries are color camera images. The kernel of the method consists of a lookup table that associates 3D surface normals with expected image brightness, modulo albedo, for a given query. This lookup table is fast to compute, and is used to render images from the models for a sum of square difference error measure. Using a data set of 42 face models and 1764 (high quality) query images under 7 poses and 6 lighting conditions, we achieve average recognition accuracy of about 83%, with more than 90% in several pose/lighting conditions, using semi-automatically computed poses. The method is extremely fast compared to those that involve finding eigenvectors or solving constrained equation systems.

This paper addresses the problem of detecting digital line components in a given binary image consisting of n black dots arranged over N N integer grids. The most popular method in computer vision for this purpose is the one called Hough Transform which transforms each black point to a sinusoidal curve to detect digital line components by voting on the dual plane. We start with a definition of a line component to be detected and present several different algorithms based on the definition. The one extreme is the conventional algorithm based on voting on the subdivided dual plane while the other is the one based on topological walk on an arrangement of sinusoidal curves defined by the Hough transform. Some intermediate algorithm based on half-planar range counting is also presented. Finally, we discuss how to incorporate several practical conditions associated with minimum density and restricted maximality.

This paper presents a new neural network structure, called Temporal-CombNET (T-CombNET), dedicated to the time series analysis and classification. It has been developed from a large scale Neural Network structure, CombNET-II, which is designed to deal with a very large vocabulary, such as Japanese character recognition. Our specific modifications of the original CombNET-II model allow it to do temporal analysis, and to be used in large set of human movements recognition system. In T-CombNET structure one of most important parameter to be set is the space division criterion. In this paper we analyze some practical approaches and present an Interclass Distance Measurement based criterion. The T-CombNET performance is analyzed applying to in a practical problem, Japanese Kana finger spelling recognition. The obtained results show a superior recognition rate when compared to different neural network structures, such as Multi-Layer Perceptron, Learning Vector Quantization, Elman and Jordan Partially Recurrent Neural Networks, CombNET-II, k-NN, and the proposed T-CombNET structure.

Fully automatic reconstruction of 3D models from images is well-known to be a difficult problem. For many applications, a limited amount of human assistance is allowed and can greatly reduce the complexity of the 3D reconstruction problem. In this paper, we present an easy-to-use method for aligning a parameterized 3D CAD model to images taken from different views. The shape parameters of the 3D CAD model can be recovered accurately. Our work is composed of two parts. In the first part, we developed an interactive tool which allows the user to associate the features in the CAD model to the features in the 2D images. This interactive tool is designed to achieve efficiency and accuracy. In the second part, 3D information extracted from different stereo views are integrated together by using an optimization technique to obtain accurate shape parameters. Some experimental results have been shown to demonstrate the accuracy and usefulness of the recovered CAD model.

Acquisition of three-dimensional information of a real-world scene from two-dimensional images has been one of the most important issues in computer vision and image understanding in the last two decades. Noncontact range acquisition techniques can be essentially classified into two classes: Passive and active. This paper concentrates on passive depth extraction techniques which have the advantage that 3-D information can be obtained without affecting the scene. Passive range sensing techniques are often referred to as shape-from-x, where x is one of visual cues such as shading, texture, contour, focus, stereo, and motion. These techniques produce 2.5-D representations of visible surfaces. This survey discusses aspects of this research field and reviews some recent advances including video-rate range imaging sensors as well as emerging themes and applications.

In this paper, the authors present general views of computer vision and image processing based on optimization. Relaxation and regularization in both broad and narrow senses are used in various fields and problems of computer vision and image processing, and they are currently being combined with general-purpose optimization algorithms. The principle and case examples of relaxation and regularization are discussed; the application of optimization to shape description that is a particularly important problem in the field is described; and the use of a genetic algorithm (GA) as a method of optimization is introduced.

The goal of this paper is to present a critical survey of existing literature on an omnidirectional sensing. The area of vision application such as autonomous robot navigation, telepresence and virtual reality is expanding by use of a camera with a wide angle of view. In particular, a real-time omnidirectional camera with a single center of projection is suitable for analyzing and monitoring, because we can easily generate any desired image projected on any designated image plane, such as a pure perspective image or a panoramic image, from the omnidirectional input image. In this paper, I review designs and principles of existing omnidirectional cameras, which can acquire an omnidirectional (360 degrees) field of view, and their applications in fields of autonomous robot navigation, telepresence, remote surveillance and virtual reality.

This paper presents a method which can effectively acquire free space on a plane for moving forward in safety by using height information of objects. This method can be applied to free space extraction on a road, and, in short, it is a road extraction method for an autonomous vehicle. Since a road area can be assumed to be a sequence of flat planes in front of a vehicle, it is effective to apply the inverse perspective projection model to the ground plane. However, conventional methods using this model have a drawback in that some areas on the road plane are wrongly detected as obstacle areas since these methods are sensitive to the error of the camera geometry with respect to the assumed plane. In order to overcome this drawback, the proposed approach named the Planar Projection Stereopsis (PPS) method supplies, to the road extraction method using the inverse perspective projection model, a contrivance for removing these erroneous areas effectively. Since PPS uses the inverse perspective projection model, both left and right images are projected to the road plane and obstacle areas are detected by examining the difference between these projected images. Because detected obstacle areas include a lot of erroneous areas, PPS examines the shapes of the obstacle areas and eliminates falsely detected areas on the road plane by using the following properties: obstacles whose heights are different from the road plane are projected to the shapes falling backward from the location where the obstacles touch the road plane; and the length of shapes falling backward depends on the location of obstacles in relation to the stereoscopic cameras and the height of obstacles in relation to the road plane. Experimental results for real road scenes have shown the effectiveness of the proposed method. The quantitative evaluation of the results has shown that on average 89. 3% of the real road area can be extracted and the average of the falsely extracted ratio is 1. 4%. Since the road area can be extracted by simple projection of images and subtraction of projected images from a set of stereo images, our method can be applied to real-time operation.

We discuss the uniqueness of 3-D shape reconstruction of a polyhedron from a single shading image. First, we analytically show that multiple convex (and concave) shape solutions usually exist for a simple polyhedron if interreflections are not considered. Then we propose a new approach to uniquely determine the concave shape solution using interreflections as a constraint. An example, in which two convex and two concave shapes were obtained from a single shaded image for a trihedral corner, has been given by Horn. However, how many solutions exist for a general polyhedron wasn't described. We analytically show that multiple convex (and concave) shape solutions usually exist for a pyramid using a reflectance map, if interreflection distribution is not considered. However, if interreflection distribution is used as a constraint that limits the shape solution for a concave polyhedron, the polyhedral shape can be uniquely determined. Interreflections, which were considered to be deleterious in conventional approaches, are used as a constraint to determine the shape solution in our approach.

The main defficulty in recognizing 3-D objects from 2-D images is matching 2-D information to the 3-D object representation. The multiple-view approach makes this problem easy to solve by reducing the problem to 2-D to 2-D matching problem. This approach models each 3-D object by a collection of 2-D views from various viewing angles and recognizes an object in the image by finding a 2-D view that has the best match to the image. However, if the size of the model database becomes large, the approach requires long time for the recognition of objects. In this paper we present a 3-D object recognition algorithm based on multiple-view approach. To reduce the recognition time, the proposed algorithm uses the coarse-to-fine process previously proposed by the authors and a genetic algorithm-based search scheme for the selection of a best matched model in the database. And, we could verify from the results of the experiments that the algorithm proposed in this paper is useful to speed up the recognition process in multiple-view based object recognition systems.

The present paper clarifies that the variance of the maximum likelihood estimator (MLE) of a parameter does not reach the Cramer-Rao lower bound (CRLB) when fitting a straight-line to observed two-dimensional data. In addition, the variance of the MLE can be shown to be equal to the CRLB only if observed noise reduces to a one-dimensional Gaussian variable. For most practical applications, it can be assumed that noise is added only to the range direction. In this case, the MLE is clearly an asymptotically effective estimator. However, even if we assume such a noise model, ML line-fitting to the data from many points of view has a high computational cost. The present paper proposes an alternative fitting method in order to provide a cost-effective unbiased estimator. The reliability of this new method is analyzed statistically and by computer simulation.

In this paper, we propose a camera calibration method that estimates both intrinsic parameters (perspective and distortion) and extrinsic parameters (rotational and translational). All camera parameters can be determined from one or more images of planar pattern consists of parallelogramatic grid points. As far as the pattern can be visible, the relative relations between camera and patterns are arbitrary. So, we have only to prepare a pattern, and take one or more images changing the relative relation between camera and the pattern, arbitrarily; neither solid object of ground truth nor precise z-stage are required. Moreover, constraint conditions that are imposed on rotational parameters are explicitly satisfied; no intermediate parameter that connected several actual camera parameters are used. Taking account of the conflicting fact that the amount of distortion is small in the neighborhood of the image center, and that small image has poor clues of 3-D information, we adopt iterative procedure. The best parameters are searched changing the size and number of parallelograms selected from grid points. The procedure of the iteration is as follows: The perspective parameters are estimated from the shape of parallelogram by nonlinear optimizations. The rotational parameters are calculated from the shape of parallelogram. The translational parameters are estimated from the size of parallelogram by least squares method. Then, the distortion parameters are estimated using all grid points by least squares method. The computer simulation demonstrates the efficiency of the proposed method. And the results of the implementation using real images are also shown.